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Homeschooling is an alternative education method 
that can be chosen by parents to educate their 
children. There is not a set age for children to start a 
homeschooling program of education. Parents choose 
to homeschool for many reasons. They may opt to 
homeschool for religious reasons. In fact, religious 
affiliation is a major factor among home educators 
(Ray, 1997). Parents may also choose to homeschool 
because their child has a hard time in a public school 
learning environment. In general, families choose to 
homeschool one or more of their children for a 
variety of reasons. According to a U.S. Department 
of Education study, parents of home-schooled 
children stated that the most important reasons for 
homeschooling their children were giving a better 
education at home, religious reasons, and poor 
learning environment at school. The U.S. Department 
of Education estimated the number of homeschooled 
children, in 1999, to be between 709,000 and 
992,000 (Bielick, Chandler, and Broughman, 2001). 

It has been shown that homeschoolers 
performed better than non-homeschoolers on 
standardized tests in the past. This is a known fact 
according to the published literature so far. The 
National Home Education Research Institute 
(NHERI) web site (www.nheri.org), has many 
references, from many published studies, stating that 
fact. In their fact sheet I, it is stated that home 
educated students scored, on average, at or above the 
80 th percentile in all areas on standardized 
achievement tests in one study. 

In NHERI fact sheet IIIc, a study of 16,3 1 1 
home educated students found that home educated 
students performed above average on the Iowa Test 
of Basic Skills, which is a test of abilities in reading, 
language and math. Here is the table of national 
percentile performance as presented by NHERI on its 
web site: 



Reading 

Language 

Math 

Conventional 

Schools 

50 

50 

50 

Home Education 

79 

73 

73 


In 1992, Calvery et al. analyzed achievement of 
Arkansas homeschooled and public schooled 
students in grades 4, 7, and 10. Homeschooled 
students were found to have scored higher than their 
public school counterpart except for grade 10 
language scores, where homeschooled students 
scored significantly lower than public school 
students. 

Research by Ray (1997) yielded results that 
indicate higher performance by homeschooled 
students. For instance, Ray reported that 
homeschooled students performed at the 87 th 
percentile in reading and 82 nd percentile in math 
compared to a national average of 50 th percentile. 
Research by Rudner (1999) indicated that 24.5% of 
homeschooled students “ are enrolled one or more 
grades above their nominal grade.”. Rudner used the 
IOWA test of basic skills (ITBS) and the tests of 
achievement and proficiency (TAP) to compare 
achievement for homeschooled and non- 
homeschooled students. His results showed that 
homeschooled students outperformed non- 
homeschooled students across all grade levels. Most 
results of the analysis showed that homeschooled 
students performed at the 75 th to 85 th percentile range 
compared to non-homeschooled students. 

Homeschooled students did well on the 1998 
ACT college entrance exam. Their average 
composite score was 22.8, about 0.38 standard 
deviation above a national average of 21. This result 
placed them at the 65 th percentile of all ACT test 
takers of 1998 (Rudner, 1999). 
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Historically, homeschooled students seem to 
have outperformed, on average, non-homeschooled 
students. However, in recent years, more people are 
choosing to go into homeschooling for their children 
for one reason or another, and this may have changed 
the demographics of home educated students in a 
manner that impacted the differences of performance 
on standardized tests between the two groups. But 
how much change is there in regard to performance 
on standardized tests is there? To answer this 
question in part, two datasets of response vectors for 
homeschooled and for non-homeschooled children 
for the same form of an ACT mathematics test were 
obtained. Next, two specific subsets were extracted 
controlling for grade level, gender, ethnicity, and 
SES (socioeconomic status). The final output was 
two subsets of examinees, (N=1477); one for the 
homeschooled group and one for the non- 
homeschooled group. A comparison of the two 
subsets was done to see if the historical trends of 
standardized tests still hold. 

It is worth mentioning here that the national 
norms are largely controlled by the scores of public 
school students. The current study does not try to 
establish any comparison between public school 
students’ results on ACT and their homeschooled 
counterparts. Rather the goal was to take 
homeschooled students and compare their test results 
to the group of all other students. It is true that the 
second group results will be largely influenced by 
public school students’ results. But it is also true that 
the second group includes students who do not 
belong to public education (i.e., private schools). 
Simply put, the goal of this study was to compare 
homeschooled students to everybody else, with 
regard to their scores on an ACT mathematics 
achievement test. The research questions that 
motivated this study were: 

1 . Is the test equally “reliable” for 
homeschooled and non-homeschooled students? 

2. Which group had a higher raw test score? 

3. If we condition on equivalent raw test scores 
for the two groups, is the pattern of responses to a 
specific item on the test similar. That is, is their a 
statistically significant difference between the two 
groups of examinees who have equal abilities (equal 
raw test scores) with regard to their pattern of 
responses to the same item on the test? 

Methodology 

During the beginning of 2003, ACT Inc. was 
contacted to obtain two data sets of response vectors, 
for the same form of an ACT mathematics test for 
homeschooled and non-homeschooled students with 


four demographics included in the data (grade level, 
gender, ethnicity, and SES). Homeschooled students 
were defined as such if they indicated on their 
application form to take the test that they were 
homeschooled. The length of time being 
homeschooled was not considered in this study. Data 
were not available in this regard. ACT was very 
helpful and provided the requested datasets in March 
of 2003. It is worth mentioning here that ACT 
provided clean datasets with no missing cases. It is 
also worth mentioning that a breakdown of the test 
items by competency was not available. According to 
the data description file from ACT (2003), the test is 
a 60-item test that takes 60 minutes for students to 
finish. The test is 

. . . designed to assess the mathematical reasoning 
skills that students have typically acquired in 
courses taken up to the beginning of grade 
twelve. The test presents five-option multiple- 
choice items that require students to use their 
mathematical reasoning skills to solve practical 
problems in mathematics. Knowledge of basic 
formulas and computational skills are assumed as 
background for the problems, but memorization 
of complex formulas and extensive computation 
are not required. The material covered on the test 
emphasizes the major content areas that are 
prerequisite to successful performance in entry- 
level courses in college mathematics. Six content 
areas are included: pre-algebra, elementary 
algebra, intermediate algebra, coordinate 
geometry, plane geometry, and trigonometry. 

(P-2) 

According to the abovementioned data 
description file. The methodology followed in 
obtaining the two datasets was as follows: 

Two steps were involved in drawing these 
samples. First, the data file for the total examinee 
population for this administration of the ACT 
Assessment (N = 330,970) was filtered to 
separate the home scholars from the non- 
homeschoolers, and to remove the record of any 
examinee who neglected to indicate one or more 
of the self-reported variables. This filtering 
resulted in the 1,807 student records represented 
in home schoolers.dat and records for 274,313 
non-homeschoolers. Next, a simple random 
sample of 5,400 records was drawn from the 
modified non-homeschooler population. These 
records are represented in nonhome 
schoolers.dat. (p.2). 

Conditioning on the abovementioned four 
demographic variables, the statistical package SAS 
was used to extract two comparable subset files. A 
randomization procedure was used wherever more 
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records of similar demographics were available in 
one of the two files. For instance if there were four 
homeschoolers’ records available for a 9 th grade male 
white examinee whose family income is more than 
$100,000 and there were 10 records of similar 
characteristics in the non-homeschoolers’ records, 
randomization was done to extract four non- 
homeschoolers records to include them in the non- 
homeschoolers newly created subset file. If the 
reverse situation occurred and the homeschoolers’ 
file had more records than non-homeschoolers, then 
the randomization procedure was applied to the 
homeschoolers’ records. In the end, two comparable 
files, with regard to the four demographics were 
obtained (N=1477). In short, matched samples were 
created through random selection. Those obtained 
files were, for the most part, the ones analyzed in this 
study. Tables 1, 2 and 3 report the demographics 
breakdown of the obtained files. It was not possible 
to report the demographics breakdown in one table 
because there were four demographic variables that 
included multiple levels for each variable. Gender 
had two levels. Grade had seven levels. Ethnicity had 
nine levels. SES had ten levels. Using a one-table 
breakdown yields 1260 cells in that table (most of 
which are zeros). Because of this, and to report a 
table that shows all four demographics, data were 
collapsed to the following number of levels for each 
variable: 

Gender: remained the same with two levels 

Grade Level: collapsed into five levels (9 th 
grade, 10 th grade, 11 th grade, 12 th grade and Other) 

Ethnicity: collapsed into four levels (African 
American, Caucasian, Hispanic and other) 

SES: collapsed into different income brackets 
($30,000 or less, between $30,000 and $60,000, 
between $60,000 and $100,000, and more than 
$100,000). 

Following are the three tables (1,2, and 3) 
mentioned above. 

Analysis 

The analysis in this study followed 3 steps: 

Computation of descriptive statistics (means 
and standard deviations and reliabilities). This part of 
the analysis tries to answer questions like: What are 
the mean test scores for the two groups? What are the 
standard deviations? What is the test reliability for 
the two groups? 

1. An item p-value (proportion correct) 
comparison. This part of the analyses tries to 
investigate the performance of each group on each of 
the test items. 


2. A SIBTEST analysis. SIBTEST was used in 
this study to answer the following research question: 

3. Conditioning on equivalent test raw scores, 
are the patterns of responses to the same test 
question, from a purely “statistical” perspective, 
similar or different. SIBTEST takes the groups of 
examinees who have equivalent raw scores on the 
test (homeschooled vs. non-homeschooled) and 
analyzes their response patterns to see if they are 
statistically significantly different from each other. 
The rationale behind using this test is to see if groups 
of students who have similar math abilities respond 
to the same test question in the same manner, 
regardless of whether they were homeschooled or 
not. 

4. Expected Frequency distribution 
comparison using IRT (item response theory): this 
part of the study involved IRT expected frequency 
simulation. This investigation was done to see if the 
expected frequencies distribution is similar to what 
was actually observed in the data sets. 

Means and Standard Deviations and Instrument 
Reliability 

Table 4 gives the “number correct” means and 
standard deviations for the two groups for the 
original datasets and for the conditioned subsets 
obtained (N=1477). 



GENDER 

Total 


Male 

Female 


Grade 

9.00 

7 

5 

12 


10.00 

80 

101 

181 


11.00 

443 

480 

923 


12.00 

151 

171 

322 


HS 

Graduate 

18 

12 

30 


College 

Student 

1 

2 

3 


Other 

4 

2 

6 

Total 

704 

773 

1477 


Table 1 . Gender by grade crosstabulation. 
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Income Category (SES) 



Less 
than 18 

18- 

24 


30- 

36 

36-42 

42- 

50 

50-60 

60-80 

80- 

100 

More 
than 100 

Total 

RACE 

Black 

9 

2 

4 

■ 

1 

9 

1 

5 

1 

3 

39 


American Indian/ 
Alaskan 

2 

1 

2 

1 

0 

0 

1 

0 

0 

0 

7 


White 

66 

79 

95 

102 

139 

155 

198 

225 

119 

131 

1309 


Latino 

2 

3 

1 

1 

2 

3 

1 

3 

1 

1 

18 


Asian 

American/Pacific 

Islander 

0 

1 

3 

2 

2 

2 

2 

0 

1 

1 

14 


Puerto 

Rican/Cuban/ 

other Hispanic 

1 

2 

0 

2 

0 

0 

0 

0 

1 

1 

10 


Multiracial 

1 

3 

0 

0 

1 

2 

2 

■ 

0 

1 

14 


Other 

1 

0 

1 

0 

■ 

2 

1 

3 

1 

0 

13 


Prefer not to 
respond 

3 

2 

3 

0 

9 

11 

7 

12 

2 

4 

53 

Total 

85 

93 

109 

112 

158 

184 

213 

252 

129 

142 

1477 


Table 2. RACE * SES crosstabulation. *Numbers are in 1000’s dollars for income categories. 


The mean scores for the two groups favors non- 
homeschoolers by about two items per examinee, 
while the standard deviations are close in value to 
each other. All datasets showed good reliability 
measures. 

Proportion Correct Comparison 

Table 5 shows the p-value per item (proportion 
correct) for the two groups along with the p-value 
differences (positive values favor homeschoolers) 

In terms of p-values, there were 1 7 items where 
homeschoolers performed better than non- 
homeschoolers. There were 43 items where non- 
homeschoolers performed better than homeschoolers. 
Whereas some of those differences may seem very 
small (i.e. 0.008), some others seem relatively large 
(i.e. 0.08). We need to keep in mind that those 


differences of p-values aggregate over all total score 
categories. 

The proportion correct comparison graph below 
shows that non-homeschoolers had an overall 
proportion correct values higher than their 
counterparts of homeschoolers. 

Figure 2 below represents the proportion correct 
differences across the items between the two groups. 
A value above zero means that the item proportion 
correct favored homeschoolers. A value below zero 
means the item proportion correct favored non- 
homeschoolers. 

SIBTEST Analysis 

Table 6 contains a summary output from a 
SIBTEST run to investigate DIF (differential item 
functioning) on the item level between 
homeschoolers and non-homeschoolers (N = 1477). 
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Grade 9th 10th 11th 12th other 



Gender 

M 

F 

M 

F 

M 

F 

M 

F 

M 

F 

Ethnicity 

SES 











African 

More than $100,000 

0 

0 

0 

0 

1 

2 

0 

0 

0 

0 

American 

$60,000 -$100,000 

0 

0 

1 

0 

3 

1 

0 

0 

1 

0 


$30,000 - $60,000 

0 

0 

0 

0 

1 

4 

5 

3 

2 

0 


Less than $30,000 

0 

0 

0 

0 

2 

5 

2 

3 

2 

1 

Caucasian 

More than $100,000 

1 

1 

14 

12 

45 

32 

9 

15 

2 

0 


$60,000 -$100,000 

4 

1 

23 

25 

108 

123 

31 

22 

4 

3 


$30,000 - $60,000 

1 

3 

32 

46 

172 

201 

67 

59 

8 

5 


Less than $30,000 

1 

0 

10 

13 

65 

63 

25 

52 

4 

7 

Hispanic 

More than $100,000 

0 

0 

0 

0 

1 

1 

0 

0 

0 

0 


$60,000 -$100,000 

0 

0 

0 

1 

3 

4 

0 

0 

0 

0 


$30,000 - $60,000 

0 

0 

0 

0 

0 

3 

3 

3 

0 

0 


Less than $30,000 

0 

0 

0 

0 

4 

3 

1 

1 

0 

0 

Other 

More than $100,000 

0 

0 

0 

0 

4 

1 

0 

1 

0 

0 


$60,000 - $100,000 

0 

0 

0 

1 

11 

10 

0 

1 

0 

0 


$30,000 - $60,000 

0 

0 

0 

2 

17 

20 

5 

5 

0 

0 


Less than $30,000 

0 

0 

0 

1 

6 

7 

3 

6 

0 

0 


Table 3. Grade by Gender by Ethnicity by SES Cross Tabulation For matched datasets (N=1477). 


Original Datasets 

Homeschoolers (N= 1806) 

Non-homeschoolers (N = 5400) 

Mean 

29.59 

32.28 

Standard Deviation 

11.03 

11.58 

Test Reliability* 

0.907 

0.919 

Obtained Subsets 

Homeschoolers (N= 1477) 

Non-homeschoolers (N = 1477) 

Mean 

30.19 

32.08 

Standard Deviation 

10.94 

11.16 

Test Reliability* 

0.909 

0.913 

Table 4. Test means, standard deviations, and reliabilities. * Cronbach’s alpha. 


There were 38 items exhibiting DIF. Nineteen 
items favored homeschoolers and nineteen items 
favored non-homeschoolers. This huge number of 
items exhibiting DIF stresses the fact that there is a 
marked difference of response patterns between the 
two groups on the item level. DIF is conceptualized 
as a difference in the probability of endorsing a 
keyed item response, when individuals with the same 
levels of ability possess different amounts of 
supplemental abilities that affect their responses to 
the item. The high number of items that exhibited 
DIF in this study may be indicative of the difference 
in mathematical training (i.e., teaching methods, 
teacher-learner interaction) between the two groups. 
Such a possible conclusion demands further research 
and investigation that is beyond the scope of the 
current study. An item by competency breakdown 


was not available, a fact that prevented further 
investigations into the possible reasons for the 
occurrence of DIF in many test items. 

Below are item characteristic curves (ICC) for 
the most significantly different items, with regard to 
DIF, where one of the items favored homeschoolers, 
and the other one favored non-homeschoolers. 

Figure 4 is a frequency bar graph for both 
groups for item 39 (numbers on the bars reflect the 
number of examinees who got the item correct at 
each score category): 

Note that the number of homeschoolers is 
greater than the number of non-homeschoolers for 
total test score groups less than 30. Non- 
homeschoolers numbers are greater than 
homeschoolers for total test score groups higher than 
30. 
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Item 

Home 

Nonhome 

H-N 

Item 

Home 

Nonhome 

H-N 

1 

0.807 

0.8429 

-0.0359 

31 

0.4631 

0.4827 

-0.0196 

2 

0.7542 

0.7583 

-0.0041 

32 

0.4482 

0.4841 

-0.0359 

3 

0.8016 

0.7962 

0.0054 

33 

0.4942 

0.52 

-0.0257 

4 

0.7942 

0.8443 

-0.0501 

34 

0.5024 

0.5714 

-0.0691 

5 

0.6723 

0.7603 

-0.088 

35 

0.4611 

0.5504 

-0.0894 

6 

0.8084 

0.824 

-0.0156 

36 

0.5741 

0.5464 

0.0278 

7 

0.7461 

0.7549 

-0.0088 

37 

0.4645 

0.5112 

-0.0467 

8 

0.717 

0.7949 

-0.0779 

38 

0.4028 

0.4035 

-0.0007 

9 

0.7949 

0.7712 

0.0237 

39 

0.5647 

0.497 

0.0677 

10 

0.5904 

0.7021 

-0.1117 

40 

0.8301 

0.7955 

0.0345 

11 

0.7536 

0.7569 

-0.0034 

41 

0.3318 

0.3927 

-0.0609 

12 

0.7387 

0.7231 

0.0156 

42 

0.2525 

0.4177 

-0.1652 

13 

0.5931 

0.6581 

-0.065 

43 

0.3907 

0.4469 

-0.0562 

14 

0.7041 

0.7163 

-0.0122 

44 

0.3731 

0.4651 

-0.0921 

15 

0.5254 

0.6012 

-0.0758 

45 

0.4685 

0.4854 

-0.0169 

16 

0.6242 

0.65 

-0.0257 

46 

0.4617 

0.4367 

0.0251 

17 

0.6953 

0.6649 

0.0305 

47 

0.3324 

0.3785 

-0.046 

18 

0.5775 

0.6229 

-0.0454 

48 

0.2241 

0.2803 

-0.0562 

19 

0.7292 

0.6676 

0.0616 

49 

0.2559 

0.3209 

-0.065 

20 

0.5674 

0.65 

-0.0826 

50 

0.3074 

0.39 

-0.0826 

21 

0.6466 

0.7129 

-0.0664 

51 

0.2735 

0.2505 

0.023 

22 

0.4265 

0.4875 

-0.0609 

52 

0.2627 

0.3074 

-0.0447 

23 

0.629 

0.6215 

0.0074 

53 

0.3453 

0.304 

0.0413 

24 

0.4983 

0.5755 

-0.0772 

54 

0.1984 

0.2404 

-0.042 

25 

0.5978 

0.6418 

-0.044 

55 

0.2241 

0.285 

-0.0609 

26 

0.5586 

0.5531 

0.0054 

56 

0.2268 

0.2092 

0.0176 

27 

0.6215 

0.6134 

0.0081 

57 

0.1672 

0.2464 

-0.0792 

28 

0.4827 

0.564 

-0.0812 

58 

0.2167 

0.1652 

0.0515 

29 

0.4455 

0.5579 

-0.1124 

59 

0.1611 

0.1313 

0.0298 

30 

0.4834 

0.5261 

-0.0427 

60 

0.13 

0.1462 

-0.0162 


Table 5. Proportion correct by item for homeschoolers and non-homeschoolers. 


Let’s inspect one item that showed DIF favoring 
non-homeschoolers. Figure 5 below is the ICC for 
the two groups, followed by a frequency bar graph 
(Numbers on the bars reflect the number of 
examinees who got the item correct for each group at 
each score category). 

Note that the pattern is still the same. There are 
more homeschoolers than non-homeschoolers for 
score categories less than 30, while there are more 
non-homeschoolers than homeschoolers for score 
categories higher than 30. This is a pattern that is 
noticeable for all the items of this test, regardless of 


whether the item showed an overall percent correct 
favoring any of the two groups, and regardless of 
whether the item displayed DIF or not. One can say, 
here, that for students with higher mathematical 
abilities, there are more non-homeschoolers that 
perform better than homeschoolers. By the same 
token, one can say that, for students with average 
and low mathematical abilities, there are more 
homeschoolers than non-homeschoolers. Keep in 
mind that this result was obtained after conditioning 
on all four demographic characteristics, with the 
same number of examinees for the two groups. 
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Run no. 

Beta-uni 

SIB-uni SIB-uni Mantel- 

z-statistic p-value Haenszel Chisqr 

p-value 

Delta 

(D-DIF) 

run no. 

Beta-uni 

SIB-uni SIB-uni Mantel- 

z-statistic p-value Haenszel Chisqr p-value 

Delta 

(D-DIF) 

1 

0.02 

1.127 

0.26 

1.75 

0.19 

-0.34 

31 

-0.02 

-1.17 

0.24 

1 

0.32 

0.2 

2 

0.03 

-1.85 

0.06 

1.78 

0.18 

0.3 

32 

-0.01 

-0.33 

0.74 

0.04 

0.85 

0.05 

3 

-0.03 

-2.39 

0.02 

4.74 

0.03 

0.53 

33 

-0.01 

-0.45 

0.65 

0.2 

0.65 

0.09 

4 

0.02 

1.799 

0.07 

3.5 

0.06 

-0.5 

34 

0.03 

1.586 

0.11 

2.56 

0.11 

-0.34 

5 

0.05 

3.606 

0.00 

12.55 

0.00 

-0.85 

35 

0.05 

3.112 

0.00 

9.74 

0.00 

-0.61 

6 

-0.01 

-0.4 

0.69 

0.18 

0.67 

0.12 

36 

-0.07 

-3.89 

0.00 

15.14 

0.00 

0.77 

7 

-0.02 

-1.55 

0.12 

1.35 

0.25 

0.27 

37 

0.01 

0.329 

0.74 

0.08 

0.78 

-0.06 

8 

0.05 

3.37 

0.00 

12.43 

0.00 

-0.83 

38 

-0.04 

-2.11 

0.04 

2.59 

0.11 

0.32 

9 

-0.06 

-4.54 

0.00 

16.74 

0.00 

1.03 

39 

-0.12 

-7.02 

0.00 

45.62 

0.00 

1.38 

10 

0.08 

4.777 

0.00 

22.64 

0.00 

-1.01 

40 

-0.07 

-4.96 

0.00 

21.81 

0.00 

1.17 

11 

-0.03 

-2.42 

0.02 

5.88 

0.02 

0.58 

41 

0.05 

2.821 

0.01 

6.97 

0.01 

-0.49 

12 

-0.05 

-3.16 

0.00 

10.15 

0.00 

0.71 

42 

0.13 

8.48 

0.00 

69.75 

0.00 

-1.87 

13 

0.04 

2.061 

0.04 

3.49 

0.06 

-0.37 

43 

0.03 

1.542 

0.12 

3.09 

0.08 

-0.34 

14 

-0.02 

-0.94 

0.35 

0.45 

0.50 

0.15 

44 

0.05 

3.18 

0.00 

11.06 

0.00 

-0.71 

15 

0.04 

2.571 

0.01 

5.17 

0.02 

-0.46 

45 

-0.01 

-0.35 

0.73 

0.05 

0.82 

0.05 

16 

-0.01 

-0.76 

0.45 

0.23 

0.63 

0.11 

46 

-0.06 

-3.8 

0.00 

13.77 

0.00 

0.74 

17 

-0.07 

-4.77 

0.00 

21.19 

0.00 

1.01 

47 

0.01 

0.794 

0.43 

0.52 

0.47 

-0.15 

18 

0.00 

0.173 

0.86 

0.05 

0.82 

-0.06 

48 

0.02 

1.695 

0.09 

2.59 

0.11 

-0.39 

19 

-0.10 

-6.49 

0.00 

39.22 

0.00 

1.38 

49 

0.03 

2.169 

0.03 

4.58 

0.03 

-0.47 

20 

0.04 

2.378 

0.02 

6.26 

0.01 

-0.55 

50 

0.05 

3.096 

0.00 

8.29 

0.00 

-0.61 

21 

0.04 

2.543 

0.01 

5.51 

0.02 

-0.5 

51 

-0.05 

-3.01 

0.00 

7.27 

0.01 

0.57 

22 

0.04 

2.177 

0.03 

4.84 

0.03 

-0.4 

52 

0.03 

1.492 

0.14 

1.84 

0.17 

-0.28 

23 

-0.06 

-4.3 

0.00 

16.07 

0.00 

0.94 

53 

-0.07 

-4.31 

0.00 

17.6 

0.00 

0.85 

24 

0.04 

2.436 

0.02 

7.43 

0.01 

-0.52 

54 

0.02 

1.381 

0.17 

3.28 

0.07 

-0.42 

25 

0.01 

0.344 

0.73 

0.15 

0.70 

-0.09 

55 

0.04 

2.687 

0.01 

7.13 

0.01 

-0.58 

26 

-0.05 

-2.72 

0.01 

6.41 

0.01 

0.5 

56 

-0.04 

-2.9 

0.00 

7.06 

0.01 

0.61 

27 

-0.05 

-3.08 

0.00 

8.16 

0.00 

0.59 

57 

0.06 

4.431 

0.00 

16.51 

0.00 

-0.95 

28 

0.04 

2.351 

0.02 

6.6 

0.01 

-0.5 

58 

-0.09 

-6.92 

0.00 

40.57 

0.00 

1.63 

29 

0.07 

4.389 

0.00 

18.78 

0.00 

-0.88 

59 

-0.05 

-3.97 

0.00 

13.43 

0.00 

0.98 

30 

0.00 

-0.04 

0.97 

0 

0.97 

0 

60 

0.01 

1.088 

0.28 

0.39 

0.53 

-0.17 


Table 6. SIBTEST run results. 
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Expected Frequencies of Scores 

Item response theory (IRT) was used to get the 
expected frequency distributions for the two groups 
using the following procedure: 

1 . The program BILOG was run twice to obtain 
ability, difficulty and discrimination parameters’ 
estimates, using a three-parameter logistic model. 
According to this model, the probability of correct 
response, P(0), for item i and subject j is given by: 


(i - a ) 


Pi (9) = Ci + j + -1.7 *ai*(0j-bi) 


( 1 ) 

Where ai, bi, and Ci are the discrimination, difficulty 
and guessing parameters. 

2. Homeschoolers’ ability estimates and item 
parameters’ estimates were transformed to the metric 
of ability estimates for non-homeschoolers using a 
mean/sigma method (Hambleton, Swaminathan, and 
Rogers, 1991). To transform the ability estimates 
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(thetas) for homeschoolers to the non-homeschoolers, 
the following formula was used: 

9y = a 9x + p (2) 

Where 9x and 9y are the ability estimates (thetas) for 
homeschoolers before and after the transformation, 
and the values of a and P above were determined 
using the following formulas: 

a = cjbj / abi (3) 

Where a is the standard deviation, and bj and bi are 
the difficulty parameters for non-homeschoolers 
and for homeschoolers. 
and 

P = p bj - a p bi (4) 

Where p bj and p bi are the mean “b” values 
(difficulty parameters) for non-homeschoolers and 
for homeschoolers. 

This transformation was done using the program 
RESCALE. (Ackerman, 2004) 

3. Finally, using the rescaled item parameters 
and latent ability estimates, the true score, x, for 
each subject was calculated according to: 

x = L,e„,P(9) (5) 

Where the probability of correct response, P(9), for 
item i and subject j is given by the three parameter 
logistic formula mentioned above. True scores were 
rounded off to the nearest whole number to obtain 
the expected number correct score. 

Expected frequencies for all estimated abilities were 
generated using the program EXPFREQ2 
(Ackerman, 2004). Below is a frequency histogram 


of the results. As one can see again, there are more 
homeschoolers than non-homeschoolers in score 
categories below 30. The situation is reversed for 
score categories above 30. 

Conclusions 

There are three distinct results from this study: 

On average, non-homeschoolers performed better 
than homeschoolers, by about two items, out of sixty 
items, on the ACT mathematics test that was 
analyzed. 

Comparing the two groups of examinees 
indicated a high number of items displaying DIF. 

This result may be due to the different 
teaching/learning media used in teaching each of the 
two groups, to different teacher/student interaction, 
or to the number of years homeschooled before 
taking the ACT mathematics test. More investigative 
research is needed in this regard. 

In general, there are more homeschoolers, who 
got a specific item correct than non-homeschoolers 
for total score categories less than 30. There are more 
non-homeschoolers who got a specific item correct 
than homeschoolers for total score categories more 
than 30. 
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Ideas for Further Research 

This study indicates a need to analyze more 
datasets on standardized achievement tests to 
compare homeschoolers to non-homeschoolers in 
general and to public schoolers in particular. There is 
certainly a wide array of standardized achievement 
tests that can be used for analysis. Comparisons can 
include ACT datasets for content areas other than 
mathematics to compare groups (i.e. English, AP 
tests, etc.). There is also a need to investigate how the 
demographics of homeschooled students has changed 
in the last 15 years and to relate those changes to 
results of standardized tests. 

Another area that calls for more research is to 
investigate why DIF occurs on the item level when 
homeschoolers are compared to non-homeschoolers 
in this mathematics achievement test. The current 
research is limited to one test and two data sets. 
Hence, DIF analysis results cannot be generalized to 
other tests. This study indicates that more research is 
needed in this regard. 

References 

Ackerman, T. (2004). EXPFREQ2 [Computer 
software]. Author 

Ackerman, T. (2004). RESCALE [Computer 
software]. Author 

ACT (personal communication, March 27, 2003). 
Bielick, S., Chandler, K., & Broughman, S. (2001), 
Home schooling in the United States: 1999. 
(NCES 2001-033). National Center for Education 
Statistics, U.S. Department of Education. 

Calvery, R., and Others (1992). The difference in 
achievement between home schooled and public 
schooled students for grades four, seven, and ten 
in Arkansas. Paper presented at the Annual 


Meeting of the Mid-South Educational Research 
Association (Knoxville, TN. November 11-13) 
Hambleton, R. K., & Swaminathan, H., & Rogers, J. 
H. (1991), Fundamentals of item response theory. 
Newbury Park, CA: Sage Publications. 

Ray, B. (1997). Strengths of their own. Salem, OR: 

National Home Education Research Institute. 
Rudner, Lawrence (1999). Scholastic achievement 
and demographic characteristics of home school 
students in 1998. Education Policy Analysis 
Archives ( No. 8) 

Shealy, R., & Stout, W. (1993). A model-based 
standardization approach that separates true 
bias/DIF from group ability differences and 
detects bias/DTF as well as item bias/DIF. 
Psychometrika, 58 (No. 2). 

Zimowski, M., Muraki, E., Mislevy, R. J., & Bock 
R. D. BILOG [Computer software]. St. 
Paul, MN: Assessment systems corporation. 

Acknowledgements 

I WOULD LIKE to thank ACT for providing the 
requested data sets. I would like to thank Professor 
Bahjat Qaqish of UNC-Chapel Hill for providing 
valuable assistance with the SAS code, and Professor 
Terry Ackerman of UNC-Greensboro for valuable 
assistance in providing software for this study and for 
reviewing earlier versions of this paper. 

I would like to thank Professors Susan Dennison and 
Jay Poole of UNC-Greensboro for reviewing this 
paper and providing many valuable suggestions. 

Also, I would like to thank Dr. Brian Ray and two 
anonymous reviewers who provided valuable 
suggestions to improve this paper. Many of their 
notes and suggestions were adopted in the final 
version of this study. ♦ 


12 



